Interaction and filling-induced quantum anomalous Hall effect in ultra-cold neutral 

Bose-Fermi mixture on hexagonal lattice 
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We investigate the quantum anomalous Hall effect in a mixture of ultra-cold neutral bosons and 
fermions held on a hexagonal optical lattice. In the strong atom-atom interaction limit, composite 
fermions composed of one fermion with bosons or bosonic holes in the mixture are formed. Such 
composite fermions have already been generated successfully in experiment [Nat. Phys. 7, 642 
(2011)]. Here we predict that this kind of composite fermions may provide a realization of the 
quantum anomalous Hall effect by tuning the atom-atom interaction or the filling of the bosons in 
the mixture. We also discuss the corresponding experimental signatures of the quantum anomalous 
Hall effect in the Bose-Fermi mixture on hexagonal optical lattice. 
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I. INTRODUCTION 

Recently, quantum anomalous Hall (QAH) effect and 
closely related topics have been attracting a great deal 
of attentions since of their fundamental interests and po- 
tential technological applications, such as new generation 
of quantum electronic devices [l|-@. In QAH effect, the 
quantized value of Hall conductance is related to a bulk 
topological number and is robust against disorder and 
other perturbations. This non-trivial topology is guar- 
anteed by the breaking of the time reversal symmetry. 
In contrast to the quantum Hall effect which happens 
at strong magnetic field and low enough temperature, 
the QAH effect is induced without any external mag- 
netic field applied to the system Q. In the past years, 
the QAH effect in solid state systems has been studied 
both theoretically and experimentally. It is theoretically 
predicted that this intrinsic quantum Hall effect is real- 
izable in semiconductor systems [H, H, Q and in graphene 
However, it has not been observed in experiment 

so far. 

On the other hand, cold atom physics has been ex- 
tended to many domains such as statistical physics, con- 
densed matter, and quantum information and provides a 
tunable artificial platform to study various novel quan- 
tum phenomena [ 141 - 1 17] . The experimental schemes to 
realize the quantum Hall effect and the quantum anoma- 
lous Hall effect by cold atoms have been proposed. Those 
methods include globally rotating the trap or optical lat- 
tice, or introducing synthet ic gaug e potential generated 
by laser beams and so on |18l - |22| . The QAH effect is 
also predicted to be realizable in the p-band optical lat- 
tice system by rotating each optical lattice site around 
its own center (23|, [HI ■ Yet, in contrast to the experi- 
mental discovery of the quantum Hall [251 ] and quantum 
spin Hall effects [III [27j, no observation of QAH effect 
in condensed-matter or cold-atom systems has been re- 
ported. 

In this paper, we present a practical scheme to realize 
the QAH in the Bose-Fermi mixture on hexagonal lat- 



tice. It is pointed out that in the strong interaction limit, 
different composite fermions can be formed in the Bose- 
Fermi mixture on optical lattice (28|. In this process, the 
free fermion combines with bosons to form the composite 
fermion when they are strongly coupled. The composite 
fermions can interact with each other on neighbor sites, 
and the strength of the interaction can be controlled by 
the filling of bosons. In a recent experiment, different 
kind of composite fermions and phase separation have 
already been observed |29|. We know that the system 
of hexagonal lattice provides a platform to study Dirac 
fermion, which leads to various novel phenomena un- 
der different conditions , including such as the topological 
none-trivial state [3CH33| • Motivated by these experimen- 
tal developments and the rich properties of Bose-Fermi 
mixture, it is naturally to wonder whether such as QAH 
effect can be realized by Bose-Fermi mixture on hexag- 
onal lattice. This scheme is indeed possible and is com- 
patible with current experiment techniques. When the 
Dirac fermion on the hexagonal lattice is strongly cou- 
pled with the bosons, the nearest neighbor (NN) and next 
nearest neighbor (NNN) interactions between composite 
fermions are generated and controlled by the filling of 
bosons. The competition of the NN and the NNN inter- 
actions drives the Bose-Fermi mixture to either the QAH 
regime or charge density wave, as pointed out in Ref. [34j . 
Differing from the previous proposed schemes based on 
the spin-orbit coupling, our proposal offers an alterna- 
tive method to realize the QAH effect in the rap idly de- 
veloping system of Bose-Fermi mixture [29|, |40| - |43| and 
provides a real system to test our understanding of the 
essence of the QAH effect. 

The paper is organized as follows. We first describe 
the model building of the Bose-Fermi mixture held on 
hexagonal lattice in Sec. II. The concept of the compos- 
ite fermion and the interaction and filling effect on the 
composite fermion is presented in the strong interaction 
limit. In Sec. Ill, the effective Hamiltonian of the com- 
posite fermion on hexagonal lattice is discussed. Based 
on the analysis of the mixture, we turn to discuss the 
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realization of QAH effect in Sec. IV. The experimental 
signatures are discussed in Sec. V. 



II. BOSE-FERMI MIXTURE ON HEXAGONAL 
LATTICE 

We consider the Bose-Fermi mixture on the hexagonal 
lattice. The Hamiltonian is written as 

Hbfh = - X)( J b,4^' + -Ii aLL + h.c.) 

m 
m) 

+ y^S^Vnijni - 1) - nm] + U ^ n i m ^ (!) 

i i 

where bi and fi represent boson and fermion annihilation 
operators on site i, and rii = b\bi and to.; = fjfi are the 
corresponding particle number operators. The first term 
represents the nearest neighbor hopping on the hexagonal 
lattice with Jp.i and Jb,i as the corresponding hopping 
parameters for fermion and boson respectively. The sec- 
ond term represents the next nearest neighbor hopping on 
the hexagonal lattice, and similarly Jf,2, Jb,2 are the cor- 
responding hopping parameters. Fig. [T]gives a schematic 
of the hexagonal lattice, which contains two sets of sub- 
lattices illustrated in the caption. The on-site interaction 
between bosons is described by V, which is assumed to be 
repulsive. The inter-species interaction between bosons 
and fermions is U , which can be either positive or neg- 
ative. There is no on-site interaction between spinless 
fermions due to the Pauli exclusion principle, [i is the 
chemical potential for bosons. The chemical potential 
for fermions is absent because the filling of fermions is 
assumed to be fixed, which equals 1/2. 

In the following, we consider the strong interaction 
limit, U, V ^> J. Rcf. [28| has investigated the case 
without assuming the geometry and dimensionality of the 
lattice. Let us first recall the main results in Ref. [28[ 
briefly. In the limit of strong on-site intra-species boson- 
boson interaction V, the particle number of the bosons 
per site in the ground state is determined by n = [Jl] + 1, 
where Jl is the dimensionless chemical potential defined 
as Jl = f-i/V and [Jl] labels the integer part of Jl. The 
strong on-site intcr-spccics boson-fermion interaction U 
leads to the formation of boson-fermion composite parti- 
cle. In the strong interaction limit, the composite parti- 
cle can be considered as an entity. It is described by the 
composite fermion annihilation operator 



or 



fi = V(n-a)!/n!(6j)Vi,«>0, 



fi = y/(n-8)l/nl(Jbi)-'fi,8< 0, 



(2) 



(3) 




FIG. 1. (Color online) Left panel: Schematic of the hexago- 
nal lattice geometry structure and the mean-filed channels for 
composite fermion. The lattice contains two triangle sublat- 
tices, labeled by A (open circle) and B (black ball). The vec- 
tor Si, i = 1, 2, 3 are the nearest-neighbor displacements from 
site B to site A. And vector bi, i = 1, 2, 3 are second-neighbor 
displacements. pA and ps denotes the particle density at site 
A and B. x& ' B is the second neighbor sites hopping driven 
by the second neighbor interaction. Right panel: Schematic 
of the composite fermions for three types: one fermion, one 
fermion with one boson, one fermion with two bosons. 



where a is the dimensionless interaction ratio U/V. It 
should be noted that s < n, which is due to the fact that 
the maximum particle number of the bosons per site is n 
in the ground state. 

There are two key parameters relevant to the physical 
results of our model and both of them could be tuned in 
experiment. One is the dimensionless chemical potential 
Ji which is defined above. And the other one is the inter- 
action ratio a — U /V. The structure of the composite 
fermion is determined by 



[a — /i + 



1. 



(4) 



where s < (s > 0) represents number of bosons (holes) 
in the composite fermion. s is determined by JI and a, 



So different composite fermions can be generated by tun- 
ing the controllable parameters Jl and a. To give a visual 
description of composite particle, we show the structure 
of the composite fermions on the Jl-a plane in Fig. [2J In 
this graph, we find that the Jl-a plane is divided into sev- 
eral blocks with different composite particle formed. On 
each of the block, the structure of the composite particles 
is independent to the interaction ratio a and dimension- 
less chemical potential Jl. For the region with a > Jl, 
all of the bosons are pushed out of the lattice site by 
fermions, i.e., there is s = n. Fig. [5] gives the phase 
diagram for ground state of the Bose-Fermi mixture for 
deep optical lattice, namely Jp, Jb — > 0. The phase di- 
agram in the weak hopping limit is independent to the 
geometry and the dimensionality of the optical lattice. 
However, more interesting phenomena appear when the 
hoppings of the atoms are turned on. After turning on 
the hoppings of the particles, the concept of the compos- 
ite particle still makes scenes if the strength of hoppings 
is much weaker than the inter-particles interaction. For 
this case, the phase of the system can also be naturally 
characterized by these blocks in Fig. [5J In addition, new 
boundaries for different phases of the system appear in 
each of the blocks, and the details depend on the geom- 
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FIG. 2. (Color online) The phase diagram for ground state of 
the Bose- Fermi mixture in the limit J — I on the Ji-a plane. 
The blocks with different colors represent the regions with 
different composite particles formed, which is determined by 
Eq. 13] For each block, the ground state of the system is 
characterized by the type of the composite particle with s 
and the number of the occupation of bosons n. The region 
marked by red dashed line is the region we considered in this 
work. 



etry and dimensionality of the optical lattice, which will 
be discussed in the following sections. 



III. THE EFFECTIVE HAMILTONIAN FOR 
COMPOSITE FERMIONS ON HEXAGONAL 
LATTICE 

In this section, we consider the effect of the hopping 
terms. In the limit J — > 0, the ground state of the system 
is Mott insulator, with each of the lattice site contains n 
bosons and at most one composite fermion. For an op- 
tical lattice with ./V sites and m spinless fermions loaded 
(the filling of bosons is determined by /i), the degeneracy 
of the ground state is N\/m\(N — m)\. In the case with 
the filling fraction of fermions equals 1/2, the degeneracy 
has the asymptotic behavior as 2 N+1 /\/N for N — » oo. 
The energy gaps for the excited levels have the order of 
U, V. Therefore the distribution on the excited states can 
be neglected for low temperature ksT <C U, V. The hop- 
ping terms provide a periodic perturbation for the sys- 
tem, and the degeneration will be lifted, and therefore 
the degenerated levels will form an energy band. The 
bandwidth will be much narrower than the energy gaps 
of these excited levels for weak hopping terms J <C U,V. 
Thus we can limit our consideration to the lowest energy 
band in this case. 

With the method of the derivation of t-J model, we 



can obtain the effective Hamiltonian limited in the low- 
est energy band. The hopping terms are assumed to be 
weak compared to the interaction strength, such that we 
could expand the effective Hamiltonian relative to the 
hopping parameters order by order. As analyzed above, 
the composite particles behave as entities, which gives 
rise to conclusion that the lowest order for the effective 
interactions between neighbor sites is 2 and the effective 
hopping terms is |s| + 1 [28] . Therefore we can estimate 
that the strength of the effective interactions are about 
J 2 /U and the effective hopping parameters are about 
jl s l+ 1 /{/l s l. After tuning on the hopping terms, the de- 
generated energy levels of the ground state of the system 
form an energy band, the bandwidth of which has the 
same order with the effective hopping parameters. Since 
the effective hopping parameters estimated asj' s ' +1 /C/' s 
is very weak for \s\ 1, the energy band is extremely 
flat and similar to case for J — > 0. The zero temperature 
approximation for the behaviors of the atoms lying on 
the lowest energy band is not proper. And the predic- 
tions based on the ground state properties of the lowest 
energy band will break down. Therefore we choose the 
region with s — in the following. The possible region 
has been marked by red dashed line in Fig. [2] 

The effective Hamiltonian for neighbor interaction can 
be obtained by the second order perturbation: 

V eff = Ue ff Y ™' ifh i + Ve *f Y ™i™h ( 5 ) 

(a) (m 

where fhi = ]}]%■ For s = 0, the effective parameters are 

U eff = ^^n(n+1), (6a) 
Veff=fyg ? n(n + 1). (6b) 

The effective interactions between the composite parti- 
cles are induced by the hopping of the bosons. As a 
result of second order perturbative expansion, the effec- 
tive interactions have the leading order J 2 /V , however, 
they are tunable via the interaction ratio a and filling of 
bosons ju. For each block with fixed s and n, the effective 
parameters are independent to the /i, while they are re- 
lated to the interaction ratio through 4a 2 /(l — a 2 ). The 
occupation of bosons n on each site is determined by Jl 
via n = [//] + 1. Hence in the process of increasing the 
filling of bosons with a fixed, the curves of U e f / and V e f / 
have step-like forms. 

The effective hopping term is given by the |s| + 1-th 
order perturbation: 

T^^-JeffYflh-IeffYflfr (7) 

For the case s — considered in this paper, the param- 
eters read as J e // = Jf,i, leff — Jf.2, which are deter- 
mined by the hopping parameters of the fermions. We 
can see that the result is also independent to the filling 
of bosons Jl. Therefore we conclude that the phase of 
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the system in each block with fixed s and n are indepen- 
dent to p. The phase boundary should be lines parallel 
to the p axes as shown in Fig. [6] This character of 
the phase diagram is valid for weak enough hoppings of 
bosons and fermions compared to the atom-atom inter- 
action. In practice, the hopping of fermions and bosons 
are finite values, and the phase boundaries might deviate 
from the parallel lines slightly. 

IV. THE QUANTUM ANOMALOUS HALL 
EFFECT IN BOSE-FERMI MIXTURE ON 
HEXAGONAL LATTICE 

The effective model given in Eq. (J7J includes a NNN 
hopping term —I e ff Ylfifj determined by hopping pa- 
rameters of fermion. It can be suppressed for weak sec- 
ond neighbor hopping Jf2- Within this constraint, the 
effective Bose-Fermi model reduces to: 

H eff = ~JeffJ2fifj + U e ff^rhifhj + V ef f £ fhiTtlj, 
(ij) (V) ««» 

(8) 

where rhi — f}fi- Due to the competition of the NN 
interaction and the NNN interaction in this effective 
Hamiltonian, the system shows up the CDW and or- 
bital ordering phase [34^36]. The mean field configura- 
tion of CDW phase is described by a density fluctuation 
P = KifhfiAt-ifLfiB)), which is shown in Fig. [TJ The 
subscript A and B denote the two triangle sublattices 
of the hexagonal lattice described in the caption of Fig. 
[TJ The orbital ordering phase is described by a complex 
second neighbor hopping parameter: (fjfj) = Xij = X% 
(Fig. [TJ). Where i,j is the second neighbor site. Since 
the system has the translational symmetry and rotational 
symmetry C3 which are invariant for different quantum 
phases, the ansatz of mean field Xij is chosen as 

= f XA = \x\e i * A ,ieA; 
Xi,i+b \ Xs = Ixl e % ^ B ,i E B. 

where b is a vector defined on the triangle sublattice. The 
mean field Hamiltonian reads as, 

Heff = -JeSfTflfi + E l V eft M + M 

+ e A J2™i + e Bj2™'i> ( 9 ) 

where ca = pU e ff, (b = ~pU e ff- The exponential fac- 
tor e l< ^ in the second term corresponds to the orbital 
ordering phase, which induces the local magnetic flux 
and breaks the time reversal symmetry spontaneously. 
The Hamiltonian of the mixture for this case is mapped 
into the Haldane model [37j- The last two terms break 
the symmetry of the two sublattices A and B due to the 
background of density wave fluctuation. The fermion has 
been assumed to be half filled. For p = and x = 0, 




FIG. 3. (Color online) (a) The spectrum corresponding to 
the semimetal state. The spectrum keeps gapless at the Dirac 
points as marked by the the red dashed circle, (b) The spec- 
trum corresponding to the QAH state. The spectrum at the 
Dirac points opens a gap when the filling of the bosons ex- 
ceeding the critical value (marked by the blue dashed circle). 
The dimensionless chemical potential for bosons p is taken as 
4.8 and the other parameters are the same with Fig. [4] 

the spectrum of the system is shown by Fig. [3] (a), 
which owns two Dirac cones. The effective interaction 
induced by the hopping of bosons might drive the sys- 
tem to the ground state with non-zero values of p and 
X- For this case, the spectrum of the system will open 
a gap at the two Dirac points as shown in Fig. [3] (b). 
The topology of the ground state is unchanged as long 
as the energy gap is kept open. Therefore, the mass 
terms at the two Dirac points are essentially important 
for the topological properties which are related to the 
appearance of QAH effect. Therefore, we try to obtain 
the linear expansion of the mean field Hamiltonian in 
Eq. @ around the two Dirac points, which read as: 
Hk/K' = ~ 3 2" kx&x — 3J ^ fl k y Oy -I- tti^~(t Z: where 

m ± = U eff p± —y-Veff |x| (sin 04 - sin^ s ) 
3 

- ^V eff \x\ (cos (f) A - COS (j) B )- (10) 

Here +, — correspond to the two Dirac points K and K' 
respectively. For p = and x 7^ (there is always 4>a=~ 
0b=7t/2), the signs for the mass terms at the two Dirac 
points are different. However for x = and p 7^ 0, the 
two mass terms have the same sign. This sign difference 
for the two cases is topologically distinctive as long as 
the energy gaps at the two Dirac points are kept open, 
and is captured by the first Chern number, which reads 
as 

Cl = \ (sign ( m+ ) ~ si g n ( m ~)) • ( u ) 

The first Chern number defined for the ground state of 
the insulator represents the Berry phase of the adiabatic 
evolution along the boundary of the first Brillouin zone 

[38| . And it gives rise to the coefficient of the Hall con- 

2 

ductor: an = C\%-. Hence the ground state with non- 
zero x processes the quantum anomalous Hall effect with 

2 

the Hall coefficient ±4r. By contrast, the ground state 
with non-zero p is a topologically trivial insulator. 

In the following, we consider the possibility of the re- 
alization of QAH in the Bose-Fermi mixture through the 
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FIG. 4. (Color online) The mass terms at the two Dirac points 
as a function of the filling of bosons Jl. The composite fermion 
at the Dirac points gains the mass when the filling of bosons 
exceeds the critical value Jl c about 4.5. The parameters used 
here (in units of E r , where E r — h 2 k^/2m is the recoil energy 
in experiment [3!|) are: Jf,i = Jb.i = 0.4, Jf,2 = 0, Jb,2 = 
0.7, U = 15, and a = 0.5. 
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FIG. 5. (Color online) The mass terms at the two Dirac 
points as a function of the interaction ratio a. The composite 
fermion at the Dirac points gains a mass when a exceeds the 
critical value a c about 0.48, and this curve stops at a s . The 
filling of bosons is chosen as Jl = 4.8 and the other parameters 
used here are the same with Fig. [4] 



mean-field analysis. The values of the order parameters 
yO, Xi 4>A and <J>b are determined by the minimum of the 
free energy. At zero temperature, the free energy equals 
the ground state energy of the system (G\H e ff\G). For 
example, we choose the interaction ratio as a = 0.5 and 
the chemical potential for bosons are tunable (as shown 
by the red line in Fig. [6]). The mass terms evaluated 
based on Eq. (|10p are shown in Fig. @] In the region 
with blue color, we give the values of the mass terms for 
the two Dirac points as functions of the filling of bosons 
Jl. For Jl > fi c « 4.5, the mass terms of the two Dirac 
points have non-zero values and process different signs. 
The Chern number is therefore equal to 1 based on Eq. 
(fTTjl . The mixture is broken to the QAH phase sponta- 
neously. Accordingly, the spectrum for the Bose-Fermi 
mixture will open a gap at the Dirac point as shown by 
Fig. El (b), which is plotted for the filling of bosons as 
ju = 4.8. The mass term increases as the filling of the 
bosons is increasing. The region with white color is not 
considered because that the composite particle in this 
region is formed with s = 1. For reasons given in Sec. 
Ill, the temperature fluctuation is much larger than the 
bandwidth for this case, hence the derivations at zero 
temperature is not suitable for this region. 

To see the effect of the interaction ratio a, we plot the 
mass term as a function of a in Fig. [S] We can see that 
the mass term obtains a non-zero value for a > a c 
0.48. For bosonic fillings belonged to the same block of 
the a-Jl plane, the critical point a c is the same. This can 
be understood as the result of the fact that the phase 
boundary of the system is parallel to the Jl axes, which 
is shown explicitly in Fig. [5] And the curves m ± (a) for 
different fillings belonged to the same block of the a-Jl 
plane coincide for reasons that the effective model in each 



block is independent to Jl (see Eq. (5])-([7])). However, for 
a > a Sl where a s is shown in Fig. [6j the system will enter 
the region with s = 1 . The discussions above concentrate 
on the region with s = and are not applicable in this 
region with s = 1. Hence the curve m ± (a) will stop at 
that point. There exist some regions of Jl where the value 
of a s is lower than the critical value a c . These regions 
could be found in Fig. [5]by moving the green dashed line 
to left. Hence, for this case, the mass term is kept zero 
until the point a s and the QAH effect would not appear. 

To give a full description of the ground state prop- 
erty, we present a phase diagram about the system for 
< a < 1 shown in Fig. [BJ The phase diagram for 
— 1 < a < is the same with this figure for reasons that, 
the effective model given in Eq. ©-([7]) for composite 
particle with s = does not depend on the sign of the 
interaction ratio a. The QAH effect is formed in the or- 
ange region, where the composite particle is formed with 
s = 0. The phase boundaries are lines parallel to the Jl 
axes. The purple region with s = 1 is not considered. 
In this work, we considered the ultra-cold atomic system 
and the spin components, which can be simulated by the 
hyperfine states of ultra-cold atoms, are not included. 
When we extend the discussions to the case including the 
internal hyperfine states, the quantum spin Hall effect is 
also expected to be realized within this scheme. 

Finally, we want to stress that the lowest energy scale 
for the observation of the QAH effect is determined by 
the bandwidth and energy gaps at the Dirac points and 
has the same order as the hopping parameter J. The 
bandwidth is determined by the effective hopping terms 
given in Eq. ([7]), which have the order J for s = con- 
sidered in this paper. As a result, the scale for the band- 
width could be estimated as J. The energy gaps driven 
by the effective neighbor site interaction are essential for 
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FIG. 6. (Color online) The phase diagram with respect to 
the interaction ratio a and filling of bosons Ji. The phase of 
the mixture for s — in the region — 1 < a < is the same 
with the case for < a < 1. The parameters used here are 
the same with Fig. [4] On the blocks with orange color, the 
QAH state is formed, with the pase boundary parallel to the 
Jl axes. The red dashed line represents the case we considered 
in Fig. [4] with a = 0.5. The green line is the schematic of 
the path for the a axes in Fig. [S] The red point denotes the 
place where the system enters the region with s = 1 and the 
corresponding interaction ratio is denoted as a s . 



the realization of QAH effect. As a result of second or- 
der of perturbative expansion, the effective neighbor site 
interaction has the order J 2 /V. However, it could be 
dramatically amplified through both the interaction ra- 
tio a = U/V and the filling fraction of bosons jS, i.e., 
the factor 4a 2 /(l — a 2 ) and n(n + 1) in the right-hand 
side of Eq. ([6]). Through tuning the interaction ratio 
a near ±1 with the method of Feshbach resonance, or 
loading bosons to the optical lattice, the effective inter- 
action could be increased quickly. Fig. [3J HI and [5] plot 
the energy spectrum and the mass terms at the Dirac 
points, from which we could find that the energy gaps 
are comparable with the bandwidth of the energy spec- 
trum. Driven by the interaction and filling effect, the 
scale for the energy gaps are boosted to become order J, 
namely, the hopping parameter. The corresponding ex- 
perimental signatures will be discussed in the following 
section. 



V. EXPERIMENTAL SIGNATURES AND 
SUMMARIES 

In this paper, we have studied the possibility of the re- 
alization for QAH effect in Bose-Fermi mixture. To make 
contact with experiments, three practical considerations 
warrant mension: (i) In order to realize the independent 
trapping of the bosons and fermions, we could make use 



of the method of constructing the species-dependent op- 
tical lattice. The frequencies for the laser beams used to 
create the species-dependent optical lattice should be the 
same to ensure the coincidence of the two optical lattices. 
Therefore we could apply two sets of laser beams with the 
same frequencies but different polarizations to construct 
the optical lattices for the fermions and bosons indepen- 
dently. In recent years, the double species trapping of 
atomic gase s [iil . [45[and species-dependent optical lat- 
tice jig |47| have been realized in experiments, (ii) The 
validity of the effective model in Eq. and ([7]) relies 
on the fact that J <C U,V. The condition can be fulfilled 
for sufficiently strong optical lattice potentials, which has 
been accessed in the experiment for the observation of the 
Mott insulator 48]. (iii) The analysis for properties of 
the ground state of the Bose-Fermi mixture at T = OK is 
valid for T much lower than the smallest energy scale in 
this system, i.e., the hopping parameters. This regime is 
accessible for sufficiently large interactions, which could 
be tuned by means of the Feshbach resonance [H, [4!| . 
In practice, the scale for the temperature fluctuation of 
the Bose-Fermi mixture ranges from nK to fiK, while the 
corresponding energy scale of the s wave scattering with 
the tuning of Feshbach resonance ranges from about //K 
to mK [H, , which covers about three orders of mag- 
nitude. Thus it is accessible, however still challenging, 
to tune the hopping strength J by changing the poten- 
tial depth of the optical lattice to fulfill the condition 
fcgT <C J *C U, V, where ks is the Boltzmann constant. 
Due to the stability of the topological properties of the 
QAH effect [5l|, the first condition for fc^T <C J is not 
as strong as the second one J <C U,V. 

Up to now, the composite fermions proposed in Ref. 
[28j have been generated in experiment [29j . The forma- 
tion of the composite particle is a key demanding of our 
proposal, and this experimental realization of it provides 
the possibility for the realization of QAH in the Bose- 
Fermi mixture on the hexagonal lattice. In their experi- 
ment, the temperature of the cloud of 174 Yb- 173 Yb Bose- 
Fermi mixture ranges from 5nK to 40nK for repulsively 
interacting cases. And the energy scale for the back- 
ground s-wave scattering of m Yb- 173 Yb atoms is about 
ks x 168nK. With the method of Feshbach resonance, 
the s-wave scattering strength could be further increased 
to fulfill the requirement of strong interaction. The de- 
tection of the QAH state in ultracold atomic systems has 
been presented extensively [52|]. The Bragg scattering 
[53| and the standard density- profile technique based on 
the Streda formula [H], HH are applicable in this experi- 
ment. 

In conclusion, we studied the QAH effect in ultra-cold 
atomic system using a mixture of bosons and fermions 
loaded on the hexagonal optical lattice. The QAH ef- 
fect is predicted to be formed when the filling of bosons 
is larger than the critical value ju c . We have discussed 
the related experimental signatures for the realization of 
the QAH effect such as the construction of the species- 
dependent optical lattice, the Feshbach resonance in the 
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Bose-Fermi mixture, and the temperature effects. Given 
the recent experimental developments on Bose-Fermi 
mixture and optical lattice, we expect that the proposal 
presented in this paper provides an alternative scheme to 
observe the QAH effect and test our understanding of its 
essence. 
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